Who Spoke What? A Latent Variable Framework for the Joint Decoding of Multiple Speakers and their Keywords

摘要

In this paper, we present a latent variable (LV) framework to identify allthe speakers and their keywords given a multi-speaker mixture signal. Weintroduce two separate LVs to denote active speakers and the keywords uttered.The dependency of a spoken keyword on the speaker is modeled through aconditional probability mass function. The distribution of the mixture signalis expressed in terms of the LV mass functions and speaker-specific-keywordmodels. The proposed framework admits stochastic models, representing theprobability density function of the observation vectors given that a particularspeaker uttered a specific keyword, as speaker-specific-keyword models. The LVmass functions are estimated in a Maximum Likelihood framework using theExpectation Maximization (EM) algorithm. The active speakers and their keywordsare detected as modes of the joint distribution of the two LVs. In mixturesignals, containing two speakers uttering the keywords simultaneously, theproposed framework achieves an accuracy of 82% for detecting both the speakersand their respective keywords, using Student's-t mixture models asspeaker-specific-keyword models.